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Abstract 

The observed large mixing angles in the lepton sector may be the first signal for the 
presence of a non-Abelian family symmetry. However, to obtain the significant differences 
between the mixing of the neutrino and charged fermion sectors, the vacuum expectation 
values involved in the breaking of such a symmetry in the two sectors must be misaligned. 
We investigate how this can be achieved in models with an SU{'i) j family symmetry 
consistent with an underlying GUT. We show that such misalignment can be achieved 
naturally via the see-saw mechanism. We construct a specific example in which the vacuum 
(mis) alignment is guaranteed by additional symmetries. This model generates a fermion 
mass structure consistent with all quark and lepton masses and mixing angles. Neutrino 
mixing is close to bi-tri-maximal mixing. 



1 Introduction 

The need to explain the observed pattern of quark and lepton masses and mixing angles remains a 
central issue in our attempt to construct a theory beyond the Standard Model. Perhaps the most 
conservative possible explanation is that the symmetry of the Standard Model is extended to 
include a Grand Unified and / or family symmetry which order the Yukawa couplings responsible 
for the mass matrix structure. 

If one restricts the discussion to the quark sector it is possible to build quite elegant examples 
involving a spontaneously broken family symmetry which generates the observed hierarchical 
structure of masses and mixing angles. However, attempts to extend this to the leptons has 
proved very difficult, mainly because the large mixing angles needed to explain neutrino oscil- 
lation are quite different from the small mixing angles observed in the quark sector. Indeed the 
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present experimental situation is consistent with the Harrison, Perkins and Scott "bi-tri-maximal 
mixing" scheme |T| in which the atmospheric neutrino mixing angle is maximal {sin'^{6@) = 1/2) 
and the solar neutrino mixing is tri-maximal {sin'^{9Q) = 1/3)). If the mixing indeed proves to be 
close to bi-tri-maximal mixing it will strongly suggest that the family symmetry is non-Abelian, 
because equality between mixing angles involving different families requires a symmetry relat- 
ing the magnitude of the Yukawa couplings of these families, something an Abelian symmetry 
cannot do. 

In this paper we discuss how bi-tri-maximal mixing can emerge in a theory with an underlying 
SU{3) f family symmetry or with a non-Abelian discrete subgroup of SU{3)f. This is of particular 
interest as SU{3)f is the largest family symmetry that commutes with 5*0(10) and so fits nicely 
with promising Grand Unified extensions of the Standard Model. However, by itself, SU{3)f 
does not explain why the mixing angles are small in the quark sector while they are large in the 
lepton sector. If it is to be consistent with an underlying spontaneously broken family symmetry 
there must be a mismatch between the symmetry breaking pattern in the quark and charged 
lepton sectors and the symmetry breaking pattern in the neutrino sector. In the quark sector and 
charged lepton sectors the first stage of family symmetry breaking, SU (3) / SU (2) /, generates 
the third generation masses while the remaining masses are only generated by the second stage of 
breaking of the residual SU{2)f. However, in the neutrino mass sector the dominating breaking 
must be rotated by ir/A relative to this, so that an equal combination of Ur and i/^ receives mass 
at the first stage of mass generation. The subsequent breaking generating the light masses must 
also be misaligned by approximately the tri-maximal angle in order to describe solar neutrino 
oscillation. 

There has already been significant progress in constructing models with a non-Abelian family 
symmetry capable of generating bi-tri-maximal mixing. These models are based on a sponta- 
neously broken A4 discrete symmetry. Of course, as just discussed, these models also have to 
arrange the mismatch of symmetry breaking in the charged and neutral sectors. In j2]. Ma 
demonstrated that, assuming such a pattern of symmetry breaking, there is left an unbroken 
discrete subgroup of A4 which guarantees the bi-tri-maximal mixing structure. Altarelli and 
Feruglio P| have recently analysed the scalar potential for a particular A4 model and shown 
that, at least in a five dimensional version of the model, it is possible to achieve naturally the 
desired pattern of symmetry breaking. 

It might seem that these A4 based models provide examples of the class of discrete subgroups 
of the SU{3)f family symmetry discussed above^. However, in these models the A4 is not 
a subgroup of the SU{3)f family symmetry which commutes with SO{10) because the left- 
handed leptons are assigned to different A4 representations to those assignments for the charge 
conjugate of the right-handed leptons. While this is perfectly possible it does mean that there is 
no straightforward way to embed these models in a Grand Unified structure and so some of the 
attractive features of such a structure are lost. In particular in the model of [2] the quarks remain 
massless at the stage the bi-tri-maximal mixing has been generated for the leptons. The quark 
masses and mixing angles appear only after further symmetry breaking and so are completely 
unrelated to the lepton mass structure. A possible exception to this approach is ^4J, which does 
assign all leptons to the same representation of A4 and can still generate maximal atmospheric 
angle. 

is isomorphic to the dihedral group A12 and is a discrete subgroup of 5*0(3) as well as SU{3). 
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Given this, we consider it interesting to ask whether bi-tri-maximal mixing can emerge in a 
theory with an underlying SO{10) ® SU{3)f symmetry ^ , capable of preserving the phenome- 
onologically successful GUT relations between quark and lepton masses. The model is heavily 
constrained because the SU{3) / multiplet assignment of all the quarks and leptons must all be 
the same. Nevertheless we show that it is possible to build a model capable of describing all quark 
and lepton masses and mixing angles in which bi-tri-maximal mixing emerges naturally. In this 
model there is a close relation between quark and lepton masses and the Georgi-Jarlskog relation 
between charged lepton and quark masses [6^ is readily obtained. The symmetric structure of the 
mass matrices by the multiplet assignments also allows it to reproduce the phenomenologically 
successful Gatto-Sartori-Tonin (GST) relation [7] for the (1,2) sector mixing. 

While completing this paper we received a paper by King [Hj who shows how to achieve 
bi-tri-maximal structure using 5*0(3) as the family symmetry. The model is similar in general 
structure to the one presented here both implementing the general strategy explored in p.[TU]. 
The main difference, apart from the different choice of family symmetry group, is that the model 
of [S] assigns left-handed states to different 5*0(3) representations from those of the charge 
conjugate of the right-handed states, generating non-symmetric mass matrices. As a result the 
GST relation is lost and the model does not straightforwardly extend to an underlying 50(10) 
unification. 

As discussed above one of the main difficulties in realising bi-tri-maximal mixing in a SU (3) / 
family symmetry model is the need to explain why the dominant breaking leading to the gener- 
ation of third generation masses in the quark sector is not the dominant effect in the neutrino 
sector too. At first sight it appears quite unnatural. However, if neutrino masses are generated 
by the see-saw mechanism ^2] in fact it can readily arise even if all quark and lepton Dirac 
mass matrices, including those of the neutrinos, have similar forms up to Georgi-Jarlskog type 
factors. To see this consider the general form of the see-saw mechanism 



where M^, is the mass matrix for the light neutrino states, is the Dirac mass matrix coupling 
p to u'^ and Mm is the Majorana mass matrix coupling u'^ to v'^. We consider the case where the 
Majorana mass matrix also has an hierarchical structure of the form 



For a sufficiently strong hierarchy this gives rise to sequential domination JSj in which the 
heaviest of the three light eigenstates gets its mass from the exchange of the lightest right- 
handed singlet neutrino with mass Mi. In this case the contribution to the light neutrino 
mass matrix of the field responsible for the dominant (3, 3) terms in the Dirac mass matrices is 
suppressed by the relative factor M1/M3 and may readily be subdominant in the neutrino sector. 
The message from this is that any underlying quark-lepton symmetry is necessarily broken in 
the neutrino sector due to the Majorana masses of the right-handed neutrino states and, through 
the see-saw mechanism, this feeds into the neutrino masses and the lepton mixing angles. This 

^See |S] for a review of previous work on SU (3) family symmetry, including original references 
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example illustrates how this effect can hide an underlying quark-lepton symmetry in the Dirac 
mass sector. 

In this paper we implement this structure to build a model with near bi-tri-maximal mixing. 
We consider only the case of a supersymmetric extension of the Standard Model because only in 
these models is the hierarchy problem associated with a high-scale GUT under control. Rather 
than work with a complete 50(10) (8> SU{3)f theory (which, in a string theory, may only be 
relevant above the string scale) we consider here the case where the gauge symmetry is Gps ® 
SU{3)f where Gps is the Pati-Salam group Gps = SU{A)ps ® SU{2)l ® SU{2)r. The 5f/(3)/ 
representation assignments are chosen in a way consistent with this being a subgroup of 5*0 (10)® 
SU{3)f. The construction of the model closely follows that of jHj and [TU], and proceeds by 
identifying a simple U{1) (g) f^'(l) symmetry capable of restricting the allowed Yukawa couplings 
to give viable mass matrices for both the quarks and leptons. We pay particular attention to 
an analysis of the scalar potential which is responsible for the vacuum alignment generating 
bi-tri-maximal mixing. 

The Majorana mass matrices are generated by the lepton number violating sector, and we find 
it to be helpful that the dominant contribution to the Majorana mass matrix for the neutrinos is 
aligned along the 3rd direction, as is the case for the fermion Dirac matrix. The major difference 
is the ordering of the hierarchical structure in the two sectors. As we will show, in this case it is 
possible to achieve bi-tri-maximal mixing very closely, with deviations coming from the charged 
lepton sector. This type of situation is described in some detail in jTTj. 

The organisation of the paper is as follows. In Section |21 we introduce the symmetry assign- 
ment of the states of the Standard Model together with the fields needed to implement a viable 
pattern of symmetry breaking. We discuss how this symmetry breaking leads to an effective low 
energy theory of fermion masses through the Froggatt-Nielsen mechanism jH] In Section 

EJwe list the dominant Yukawa couplings in the effective theory responsible for fermion masses 
and compute the Dirac and Majorana mass matrices. The details of the symmetry breaking 
alignment are presented in the Appendix where we discuss the details of the minimisation of 
the effective potential, particularly the minimisation and effect of the D-terms and the effect of 
the soft SUSY breaking masses. In Section |^ we discuss the phenomenological implications of 
this model, and estimate the magnitude of the corrections to bi-tri-maximal mixing. Finally in 
Section El we present the conclusions. 

2 The construction of SU{3)f family symmetry models 
2.1 Symmetries 

As discussed above we will start with the gauge group Gps (8> SU{3)f. We wish to assign our 
states to representations in a manner consistent with an underlying 5*0(10) ® SU{3)f symmetry 
so we will discuss the representation content as if this is the gauge group even though we will use 
only the 5[/(4) ps (S> SU{2) l ® SU{2)p® SU{3) f subgroup in constructing models. The Standard 
Model (SM) fermions ipi, ipj ^ are assigned to a (16, 3) representation of 50(10) (g) SU (3) /. The 
Higgs doublets of the SM are part of a (10, 1) representation, H. In addition we introduce an 
adjoint of Higgs fields H^^ ~ (45, 1), which in our effective theory has a vacuum expectation 

~ 1,2,3 are family indices 
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Table 1: Model field and representation content 



value (vev) consistent with the residual Gpg® SU{3)f symmetry which leaves the hypercharge 
Y = T3R + {B- L)/2 unbroken ^. 

To recover the SM we must completely break the family symmetry. We will do so in steps, 
first with a dominant breaking from SU{3) f —>■ SU{2) f and then the breaking of the remaining 
SU{2)f. This spontaneous symmetry breaking will be achieved by additional Higgs fields that 
are either triplets (3j) or anti-triplets (3*) of the family SU{3)f, and the alignment of their 
vacuum expectation values (vevs) is a principal concern of this paper. In a realistic model it is 
necessary to extend the symmetry in order to eliminate terms in the effective lagrangian. The 
construction of a specific model requires that the full multiplet content is specified together with 
its transformation properties under Gps ® SU{3)f and under the addition symmetry needed 
to limit the allowed couplings. In the model constructed here the the additional symmetry is 
U{1) (8) U'{1). The multiplet content and transformation properties for the model are given in 
Table HJ In addition to the fields discussed above it includes the fields 9 and 9 whose vevs break 
SU{4:)ps and lepton number and generate the Majorana mass matrix. There are also additional 
SU{4:)ps ® SU{2)l (g) SU{2)p singlet fields needed for vacuum alignment as discussed in the 
Appendix. 
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2.2 Spontaneous symmetry breaking 



We now summarise the pattern of family symmetry breaking in our model. The detailed min- 
imisation of the effective potential which gives this structure is given in the Appendix. 

The dominant breaking of SU{3)f responsible for the third generation quark and charged 
lepton masses is provided by the ^3 vev 

<03> = ( 1 ) f M (1) 





where the SU (3) x SU (2)jj structure is exhibited. To preserve D-flatness, another field, ^3, also 
acquires a large vev 

(03) ^ ( S I (2) 

where 5 is a small correction needed to maintain D-flatness (c./. the Appendix), appearing once 
the remaining symmetry breaking is taken into account. Notice that 03 also breaks the SU{2)ji 
so the residual symmetry is now SU (4) ps ® SU {2)l ® ^7(1)_r. 

The breaking of the remaining family symmetry is achieved when a triplet 02 acquires the 

vev 

(02) ^ ^ j (3) 

Due to the allowed couplings in the superpotential (c./. the discussion in the Appendix) this 
vev is orthogonal to (^s). 

Further fields acquire vevs constrained by the allowed couplings in the theory. As detailed 
in the Appendix the field 023 acquires a vev 



^23 



> ~ ( 6 -6 ) (4) 



Note that it is the underlying SU (3) / that forces the vevs in the 2nd and the 3rd directions 
to be equal in magnitude, so that the 023 is rotated by n/i relative to the 03 vev. This will be 
important in generating an acceptable pattern for quark masses and in generating bi-maximal 
mixing in the lepton sector. Finally the fields 0i23 and 0i23 acquire the vevs 



n23 



> = (c c c). (5) 



(0123) = 1^ C j (6) 

where c = ce*^. 

Its important to note again that, even though SU (3) / is spontaneously broken by these vevs, 
it is also responsible for aligning them so that the elements have equal magnitude. As we shall 
see this structure is crucial in obtaining tri-maximal mixing in the solar neutrino sector. 
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3 The fermion mass sector 



3.1 The effective superpotential 

Having specified the multiplet content and the symmetry properties it is now straightforward 
to write down all terms in the superpotential allowed by the symmetries of the theory. In 
this Section we concentrate on those terms responsible for generating the fermion mass matrix. 
Since we are working with an effective field theory in which the heavy modes associated with 
the various stages of symmetry breaking have been integrated out we must include terms of 
arbitrary dimension. In practice it is only necessary to keep the leading terms needed to give all 
quarks and leptons a mass and to determine their mixing angles. For the generation of quark, 
charged lepton and neutrino Dirac masses these are 

Py = -n^^l^^^i^jH (7) 



M3 

1 -r 



M2 



M2 



<Pls^^<P{23^^H (9) 
^\23^i43r,H (10) 



+ J^m<l>{23r,HH,S3h,f (11) 

Note that the terms appear suppressed by inverse powers of a mass scale which we have 
generically denoted by M. Identification of this scale for each term is important in studying 
the phenomenology and to do this one has to identify how these non-renormalisable terms arise. 
This occurs at the stage where superheavy fields, the messenger fields, are integrated out. In 
Section IX^ we consider this in detail. 

The terms allowed by the symmetries responsible for the Majorana mass matrix involve the 
6"^ anti-triplet responsible for breaking lepton number and SU{4:)ps. Its vev is aligned along the 
3 direction (c./. the discussion in the Appendix). The leading terms are 

Pm = (12) 

^ l4^23rAr,0'c^l23j'c^3, + -^J'r^4>i3r,0'' 4^123 (13) 



+ ^,4>\23r^4>\23r^e'ct>23,e'ct>,, + ^^"Vr0i23V^,'0m023. ^"'03, + ^^"Vr0i23V^l^"V23. 0i2303. (M) 

In all these equations we have omitted the overall coupling associated with each term. These 
are not determined by the symmetries alone and are all expected to be of 0(1). 
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3.2 The messenger sector 

The scale M entering in the effective superpotential is set by the heavy messenger states in 
the tree diagrams giving rise to the higher dimension terms. There are two classes of diagram, 
corresponding to heavy messenger states transforming as 4s under SU{4)ps (vectorlike families) 
or those that don't (heavy Higgs). Which class dominates depends on the massive multiplet 
spectrum which in turn is specified by the details of the theory at the high scale. Here we 
assume that the heavy vectorlike families are the lightest and dominate. 

These states carry the same quantum numbers as one of the external quark or lepton fields. 
Due to the underlying SU{2)l, the Mq^ (the left-handed quark messenger mass scale) will be 
the same for the up and down quarks. However SU{2)ii is broken and so the messenger mass 
Muj^ (the right-handed up quark messenger mass scale) needs not be the same as M^^ (the right- 
handed down quark messenger mass scale). The lepton messenger mass scales have a similar 
structure, with M^^ (the left-handed lepton messenger mass scale) contributing equally to the 
charged lepton and neutrino Dirac couplings, but with Mg^ (the right-handed charged lepton 
and neutrino messenger mass scale) having different scales due to SU{2)ji breaking effects. 

This splitting of the masses of the messengers is important because it is responsible for 
the differences between the up and down quark and lepton masses. As we noted above, the 
underlying SO{10) ® SU{3)f ® U{1) ® U'{1) structure forces all matter states to have the same 
family charges and so the leading terms in the superpotential contribute equally to all sectors. 
However the soft messenger masses which enter the effective lagrangian are sensitive in leading 
order to SO{10) breaking effects and thus can differentiate between these sectors by fixing 
different expansion parameters in the different sectors. 

To see what choice for the messenger masses is necessary phenomenologically we note that 
a fit to the up and down quark mass matrices has the form ^3] 



Yu oc 



el 0{el 
• el Oiel 



1.5e^ 



with expansion parameters 



1-36^ 



0.15 



(15) 



(16) 



(17) 



e„ ~ 0.05, erf 

From eq ()13p it may be seen that in the quark sector the expansion parameters in the (2, 3) 
block are essentially determined by the 023 vev divided by the relevant messenger mass scale. 
If the expansion parameters are to differ it is necessary for Mq^ to be larger than the other 
messenger masses in which case 

eu4 ^ T7^ (18) 
Clearly to generate the form of eq ()17|) we then need 



M, 



dR 



Ur 



(19) 
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In the lepton sector we know that the SU{5) relation mi, ~ m^- at the unification scale is, 
after including radiative corrections, in good agreement with the measured masses. For this 
to be the case here it is necessary that the breaking of the SU{4)ps should not dominate the 
contribution to the down sector messenger masses. This is to be expected in these models 
because the breaking of SU{2)r occurs through the lepton number breaking sector which does 
not couple in leading order to the the right-handed charged lepton messenger states. With 

Men - 

Mq, ^ Ml, 

the right-handed messengers dominate and the relation mi, ~ follows. However the right- 
handed neutrino messengers do couple in leading order to the SU {2)r breaking fields and so may 
be expected to be anomalously heavy. This is helpful, because a small right-handed neutrino 
expansion parameter e^^^ naturally explains the large hierarchical structure of Majorana masses 
which, as discussed above, is needed to overcome the large Dirac neutrino mass in the (3, 3) 
direction. 

The expansion parameters in the lepton sector are then given by 

^UR,UL,eR — -jrz (20) 

Bounds on the messenger masses of the neutrinos (or rather, the associated expansion pa- 
rameters) will be presented in Section |3] The other expansion parameters are chosen to fit the 
masses as in eq ifTTj) . 

Note that the contribution to the (3, 3) entries of the quark and charged lepton mass matrices 
involves the combination a„/M„^ and ad/Md^, = ad/Mej^ for the up and down sectors respectively. 
If 03 dominates the {SU{2)r breaking) contribution to the messenger masses then we expect 
a^/Mu^ ~ ttd/Md^ — 1 which is indeed the phenomenologically desirable choice [H]. 

3.3 The Dirac mass matrix structure 

Using the expansion parameters introduced above we can now write the approximate quark mass 
matrices for the second and third generations following from eqs((Zj) and (jHl) in the form 



d 



In writing this form we have made a choice for the H^^ vev that appears in the terms 
contributing to these elements. (i?45) preserves Gps and is proportional to the hypercharge 
Y = T^R + {B — L)/2 inj, [ini- To fit the strange quark mass we take its magnitude to be such 
that 



M Md, 



0(1) 



and with this choice, the factor Y (ur) /Y {dp) = —2 appears in Y^. 

Because the charged lepton messengers have the same messenger mass scale as the down 
quarks, the charged lepton mass matrix is similar to Yd with the form 

y. « ( % -f' ) (22) 
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where the Georgi-Jarlskog factor Y {cr) jY (c^j?) = 3 comes from (-^45). This factor gives ^ 
3ms which is, after including radiative corrections, in good agreement with the measured masses, 
an obvious advantage to having an underlying GUT. 

Having explained the origin of the structure in the (2, 3) block it is straightforward to follow 
the origin of the full three generation Yukawa matrices for the quarks and leptons. Including 
the effect of the terms in egs® and (fTUI) . we have 





2 

—g@el^ed 2e, 



2e^ 



-9&^u^d 

2e?,^ 



(23) 



Yi oc 



9 ©4 
-9@4 



9@(^d 
-9@4 



9q4 
^2 



-e: 



9g4 
^4 

-^4 



-9q4 

-4 
1 



-9g4 
-3e3 



v 



9€ 





'^UL^d 



{9i 



i + 9e)4L^d 



-9&^tL^d 
(9© - 9q)4, ^d 



9&^UL^d {-9m + 9e)4rM 



\ 



(24) 



(25) 



(26) 



In this we have restored the dependence on the unknown Yukawa couplings of 0(1), after a 
suitable 0(1) redefinition of the expansion parameters. As will be discussed, we will concentrate 
on the case Qq = (7®, which is needed for the GST relation [7j. 

The structure of the Dirac neutrino mass matrix Y,y follows from the terms in eqs© and 
The form shown applies in the limit where the dominant carriers are left-handed. As 
discussed above this is to be expected because the SU{2)ii breaking gives rise to a very heavy 
right-handed neutrino messenger mass, corresponding to e^^ -C e^j^ . The term involving the -^45 

field in eq® is suppressed by an additional messenger mass. For ^ < et^ 
leaving the leading terms shown in eq lj^Uj) . This corresponds to the upper bound e 

Note that the differences between the (1, 2) and (1, 3) elements of Y^, needed to fit the data 
c.f. eq. (jl6p . come from eq (jll|) . Thus, due to H^^ they are subdominant in the neutrino matrix 
for the reason just showcased, so eq (j^ is essentially unchanged by the contribution from eq (fTT|) . 



- <C et, ed it is subdominant 



3.4 Majorana masses 

The heavy right-handed neutrino Majorana mass matrix has the largest contribution from the 
operator of eq (fT^ giving the (3, 3) component 



2 



33 



(27) 
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The terms of eqs (|13p and (|14j) give the Majorana mass matrix of the form 




where the Aj are the 0(1) factors coming from the couphngs associated with the different oper- 
ators of eqs (fT!?|l and ((T3j). 



4 Phenomenological implications 

It is now straightforward to determine the masses and mixing angles in the theory. By construc- 
tion the form of the up and down quark masses is in agreement with the phenomenological fit 
of eq ()15|) and eq ifTBj) . If we further have Qq = g@, giving a symmetric mass structure, then the 
(1, 1) texture zero gives the successful GST relation [7] relating the light quark masses and CP 
violating angle to the mixing angle in the (1, 2) sector. A symmetric form for the mass matrix 
is to be expected from the underlying 5*0(10) and we will assume this is the case here. 

As discussed above, the charged lepton mass matrix gives the phenomeno logically successful 
relations rUb ~ rrir and ~ Sm^ at the unification scale. Moreover, the (1, 1) texture zero 
implies that Det[Ye] ~ Det[Yd\ so that rrie — rrid/S at the unification scale, again in excellent 
agreement with experiment once one includes the radiative corrections to the masses. The 
contribution to the mixing angles in the lepton sector is given by 



0[2 - J— (29) 



^23 



rrir 



Finally we determine the neutrino masses and mixing angles. The Majorana mass matrix 
has mass ratios given by 



Ml 

M2 _ f ^unY ^4 



(30) 



Ms V 



d 



Due to the large hierarchy in the Majorana mass matrix between Mi, M2 and M3, the contribu- 
tion to the light neutrino masses from the exchange of the third (heavy) right-handed neutrino is 
negligible. This is despite the fact that the dominant Yukawa couplings in the Dirac mass matrix 
are to the third right-handed neutrino, and realises the stategy discussed in the introduction to 
explain the mismatch in the family symmetry breaking patterns in the charged fermions and 
neutrino sector. 
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The light neutrino masses are given by 



m@ = -J^ (31) 



..e ^ (32) 



= '"^3 ^^^^ 

where /i is the doublet H Higgs vev generating the up sector masses, and where we have absorbed 
the 0(1) couplings in a redefinition of the Majorana masses. Up to these 0(1) terms the ratio 
between the neutrino masses is given by 



m 



ed (34) 



1/3 / ^VR 



me \ed J " 

With our hierarchical mass structure the observed mass squared differences relevant for 
atmospheric and solar oscillations are approximately given by m| and rriQ respectively. Up to 

the 0(1) coefficients, m@ = (^—-^ ^-^id a fit to atmospheric oscillation is readily obtained 

by a choice of {j^^ With this normalisation the solar oscillations have mass squared 

difference given by m^, = e'^rriQ. With the expansion parameter given in eq ()17|l from the down 
and charged lepton mass hierarchy fits and choosing the 0(1) coefficients to be unity we obtain 
excellent agreement with the magnitude found in solar neutrino oscillation. 

The mixing angles are also readily obtained. To understand the results it is convenient 
first to neglect the off-diagonal terms in the Majorana mass matrix. The dominant exchange 
term in the see-saw mechanism is uf. From eqs© to Q we see that only couples via 
eqlQ to the combination 023 V'i °^ i^fj. ~ ^t)- As a result the most massive neutrino is close to 
bi-maximally mixed. The exchange of 1/3 is responsible for generating the next most massive 
neutrino. From eqs© to © we see that it couples by both eq© and eq lfTUj) to the combination 
{v^ — Vr) + {ve + + ^t)- Diagonalisiug the masses the effect of this term is to introduce mixing 
at O(^) in the most massive state between the combinations (z/^ — z/^) and (z/g + z/^ + Vt)- 
However we have not yet introduced the effect of the off-diagonal terms in the Majorana mass 
matrix, notably the entries 12 and 21, which also introduce such mixing. Doing so we find 
that, due to the underlying symmetry of the theory, these mixing terms cancel between the two 
contributions giving 



3 
1 

2 

sm^e\^ = 



sin' ^2^3 
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Thus we get pure bi-tri-maximal mixing in the neutrino sector together with 6'(^ = 0. Note that 
the underlying SU{3) f family symmetry is responsible for these predictions. 

Finally we should combine the contributions from the charged lepton sector to obtain the 
PMNS angles. We should stress that the actual value of the corrections arising from the charged 
leptons depends on the value of the CP violating phase of the lepton sector, as shown in jllj . 
Our model doesn't allow us to predict this phase independently (it originates from unknown 
phases of the fields involved in the vacuum alignment), but one can make a prediction of its 
value from the measured angles. Considering the cases that give us the largest deviations, we 
obtain a range of possible values for the angles given by 

sin ^12 ^ 7f(l±^i2) 

sin 023 ^ 7f(l±^23) 



sin 013 ~ :^^i2 



thus we have 



sin^^io ~ 0.024 



in good agreement with the experimentally measured values. 



5 Summary and conclusions 

In this paper we have shown how near bi-tri-maximal mixing in the lepton sector arises from 
a spontaneously broken SU{3)f family symmetry through vacuum alignment. The model con- 
structed has a phenomenologically acceptable pattern of quark and lepton masses. It generates 
the successful GST relation between the mixing angles and masses of the first two generations 
and the Georgi-Jarlskog relations between down quarks and charged leptons. The neutrino sec- 
tor generates precise bi-tri-maximal mixing and a zero value for ^13. The charged lepton sector 
generates small corrections to bi-tri-maximal mixing and a non-zero value for 613. The see-saw 
plays a very important role in explaining the difference between quark and lepton mixing angles. 
This is because the large family symmetry breaking along the 3 direction, which dominates the 
quark and charged lepton masses, is suppressed in the light neutrino sector because of the heavy 
right-handed neutrino mass in the 3 direction that appears in the see-saw mechanism. The 
model has a fairly complicated multiplet content, particularly in the symmetry breaking sector. 
However it represents only one of a large class of such models and it may be hoped that more ele- 
gant versions exist. In any case our example demonstrates the existence of a phenomenologically 
satisfactory model in this general class. 
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Appendix - Vacuum Alignment 



As discussed above the alignment of vacuum expectation values is crucial to the generation of 
bi-tri-maximal mixing. In this Appendix we consider the minimisation of the scalar potential 
and show that with the terms in the superpotential allowed by the symmetries of the theory the 
desired vacuum alignment does indeed occur. 

The field content and their symmetry properties are given in Table ^ 

We start with the 9i and 9^ fields. Their soft masses tuq and have radiative corrections 
coming from gauge and Yukawa related couplings. The gauge couplings contribute positively 
to their the soft mass squared while the Yukawa couplings contribute negatively. If the latter 
dominates their mass squared can be driven negative, triggering radiative breaking and driving 
nonzero vevs for the fields. Since 9 and 9 are in conjugate representations there is a a D— flat 
direction with equal vevs for 9 and 9. Assuming this direction is also F— fiat (this depends on 
the structure of the massive sector of the theory which is not specified here) the scale of the 
vevs is close to the point A, at which rn^{h) +m|(A) = 0. Without loss of generality the basis is 
chosen such that the breaking in the S'f/(3) / direction is aligned along the 3rd family direction. 

Turning now to the <^3 and 03 fields we consider the superpotential of the form 

P3 = X3(Tr[4]03^-M53) 

which is allowed by the symmetries of the theory (the trace is taken to yeld the SU{3)f x 
SU{2)^ invariant component of the first term). M is a mass scale (it can arise from a singlet 
like S'3, as long as the charges are suitable). We assume that the field undergoes radiative 

m — I 1 2 m 

breaking with vev /i3. Then the 03 and 03 vevs are triggered by the F-term l-Fxal • These vevs 
develop along the D— fiat direction 

(03> = ( 1 ) ® ( ° ) (35) 

where the SU{3)f x SU{2)r structure is exhibited, and 




Note that these vevs align along the same direction as 9 and 9. The reason is that 9 and 9 break 
SU (3) / to SU (2) f so that the stabilising gauge radiative corrections to 03 act more on the 1st 
and 2nd family elements than the third family element. As a result the third components of 03 
and 03 are the ones that have the smallest mass squared and so it is energetically favourable for 
the vevs to develop along them. 

The 03 vev acquires a nonzero but small entry S, giving it the structure of eqQ. S is fixed 
to preserve Z^-fiatness (after the other vevs are triggered). 

Consider now the adjoint field S^, which we introduce for alignment reasons. The symmetries 
of the theory allow the following renormalisable terms in the superpotential 

= ^Tr (S^) + M^Tr [T?) (37) 
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where M is a mass scale in the effective theory associated with the ultraviolet completion of the 
theory. It could be the breaking scale of the underlying SO{10) in a GUT implementation of 
the theory or could be associated with the string scale in a string theory realisation. It arises 
from the vev of a field with correct R charge (2/3, same as that of S). This term induces a vev 
of the form 

/ a \ 

(S) = a (38) 
\ -2a / 

The relative alignment of (S) and (^s) follows because, with the vevs aligned, both 03 and S 
break SU{3)f to SU{2)f so that the stabilising radiative corrections to 03 act more on the 1st 
and 2nd family elements leaving the 3rd component lightest and thus minimising the residual 
vacuum energy. 

Consider now the fields 02 , 023- In discussing the cancellation of the D— terms we allow for 
the presence of additional fields 02, 023 transforming in conjugate representations to 02 and 023- 
The alignment of the vevs of these fields is responsible for the difference of the mixing angles 
between the quark and lepton sector. This alignment is driven by the following superpotential 
terms 

P23 = 1^2Tr[0^3]02, + X23 (0^03.0^02, - + ^23 (0^023.) (39) 

where the trace is over the SU{2)r indices and only the SU{3)f family indices are exhibited. 
The quantity fi23 is a mass scale, and similarly to fi^ it can arise from an appropriate singlet 5*23. 
Following from this superpotential are F— term contributions to the scalar potential which force 

the vevs of the form of eqQ, eq(jD). The term 1^X23!^ =|023'/'3i^23</'2j "/^tsl forces both the 

— I 1 2 I ^ ■ 1 2 

vev of 02 and of 023 to be nonzero. The term \Fy2 \ = 0302, forces the 02 vev to be orthogonal 

to that of 03. Without loss of generality we may choose a basis in which it is aligned along the 

2 direction as in eq® 



(02) =yyj 

thus 023 must have vevs in both the 2 and 3 directions 

(023) = ( b 63 ) (41) 

Consider now the pair 0i23, 0i23- The alignment of these will complete the vev alignment 
discussion. The relevant terms are 

i^l23 = ^123 (01230123. " f^us) (42) 
+Y,23 (0230123.) (43) 
+ Zi23 (0123^10123,) (44) 
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The operators in eq(|i^. eq(|iH|) and eq (jii|) trigger and align the vevs of 0i23 eq© and 0i23 
eq® through the vevs of 023, S and the mass scale fius (like with ^23, this scale can be obtained 
though the vev of a singlet S'123). 

Throughout the analysis the important effect of D-terms and soft supersymmetry breaking 
mass terms rrii must be taken into account, since these terms play an important role in determin- 
ing which vevs occur and in vacuum alignment. We will analyse these conditions perturbatively 
in an expansion involving small mass ratios, assuming the ordering mj <^ 5, c, c <^ 6, 63, ?/ -C a„, 
ad which is the phenomenologically viable range and which is readily obtained by a choice of 
the free parameters of the theory. In this case minimisation of the potential in leading order 
proceeds by setting the D and F terms to zero and minimising the contribution to the potential 
coming from the soft terms. Of course the true minimum corresponds to the case that the D and 
F terms are not zero but have a magnitude comparable to the contribution of the soft terms. 
However this involves a nonleading change in the vevs found setting the D and F terms to zero 
and so can be dropped in leading order. In this way we can establish the local structure of the 
potential. 

The interesting vacuum structure has the (D-fiat and F-flat) form 



(03> = ( 1 ) ® ( '0 







(45) 





6 I (46) 



(02> = ( ^^FTP ) (47) 

(48) 





623) = ( h -h) (49) 

(50) 
(51) 

^123) = 1^ c j e'^ (52) 

where the overall phases are factored into the definitions of h and c leaving only relative phases 

which are explicitly shown. These phases uniquely preserve the F-flatness in this configuration. 

The relative phase of 023 is connected with the relative phase of 023 due to the 
1 2 I ^ ■ 1 2 
\Fy2z\ —W22,4>22,i\ orthogonality condition. 
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The D-teim flatness conditions constrain the relative phases of the 0i23 and 0i23 components 
to be zero. 

The relative phase of the 023 field components is vr. This follows because the F— term 
l-^yi23l^ =|<^23<^i23i I forces the respective vevs to be orthogonal. 

To demonstrate that this is indeed a vacuum solution and prove the ratio of the vevs, we 
must include the effect of the soft mass terms. We assume that all the fields in discussion have 
positive mass squared. 

— — I 1 2 

Note that the vevs of 02, 023 , 02 and 023 are triggered by minimising l-Fxagl , which requires 
< ybb^ >= /i23/ a/ci^ + a^. The mass term 77223 |<^23| + ^2 \4'2f + ^^23 I023|^ + rn^ |02| is then 
minimised by 

\b\ = \bs\ 

2 _ ^23 + ^23 u2 

y 2 I - 2 

77I2 + 7772 

Note that the equality of vevs in the 2 and 3 directions is a direct result of the soft mass 
degeneracy of 023 2 and 023 3- This follows from the underlying family symmetry 77733 \ 4'2z\ = 

2 |2 |2 

"^23(1 ^23,1 1 + 1 023,2 1 + 1 023,3 1 ) showiug liow the required vacuum alignment is due to the family 
symmetry even though it is spontaneously broken. 

The remaining vacuum structure applies in a given region of soft mass parameter space. The 
correct structure is obtained if I77723I > I77723I, |mi23|, \mi2-i\ > \^2\- This ensures soft mass 
minimisation won't allow 023 to be perturbed away from the vev already determined. Then, as 
long as 02 soft mass is light, minimisation of the soft terms favours the entries of 0i23 and 0i23 
to be of equal magnitude (some were already constrained to be so from F-flatness). The D-term 
highly constrains the phases, allowing only overall phase ambiguities as shown in eqs()45|) to ()52|) . 

There is a longstanding problem associated with gauged family symmetry due to the fact 
that the D— terms typically are no n- vanishing and contribute to the soft masses of the quarks 
and leptons in a family dependent way. This is potentially disastrous as such masses can lead 
to unacceptably large flavour changing neutral currents. Following Murayama [T7j one sees that 
the effect is proportional to the difference in the mass squared of the two flelds developing large 
vevs along the D— flat direction. As a result the effect can be suppressed if these masses are 
closely degenerate. In ^ we showed how this could naturally result in an SU (3) / model and the 
same structure readily emerges here. This is not the only way the D— term contribution may be 
negligibly small. A speciflc example follows when the SUSY breaking mediator mass is less than 
the family breaking scale because the radiative graphs generating the dangerous soft masses are 
suppressed by the ratio of the two scales. This will be the case in this model for gauge mediated 
supersymmetry breaking. A more extensive discussion of the suppression of D— term soft mass 
contributions will appear shortly. 
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